Dead or defective pixels are those pixels in a video clip that consistently stand out from their neighborhood pixels. Defective hot pixels are stuck at the brighter end of the intensity range whereas defective cold pixels are stuck at the darker end of the intensity range, both incapable of properly capturing the scene color. Defective pixels in native camera content usually have very sharp edges, i.e. high contrast with their neighbors. Defective pixels in native camera content may be caused by problems with particular portions of the solid-state imaging sensors, or by dust or dirt on the sensor surface. Sometimes such defective pixels produce intensity levels that are invariant over a series of images or frames, and cause Residual Point Noise (RPN) in a recorded video. Generally, RPN spots include several numbers of dead or destroyed pixels in the horizontal and/or vertical direction that cannot reproduce colors properly. If such video is sub-sampled or processed, the defective pixels are usually blurred and not as sharp as defective pixels from the native camera output.
In theory, the luminance of such defective pixels could be significantly lower or greater than their neighboring pixels. In a dark scene, defective hot pixels may appear significantly lighter than their non-defective neighbors, and, in a bright scene, defective cold pixels may appear significantly darker than their non-defective neighbors. Also, in theory the RGB value of dead pixels never changes and each dead pixel is present from the very beginning of a scene, or shot, to the end of the scene. However, in real-world video clips, and especially video clips for broadcasting purposes, the RGB value of dead pixels could change or appear and disappear during different portions of the video. This may be due to transcoding operations, up-conversion (rescaling) operations or the nature of the video content. This can also occur when a video clip is assembled from more than one source, with one source having a first set of defective pixels and another source having a second set of different defective pixels. In such cases the resultant video may appear to have defective pixels that momentarily appear or disappear. When a video clip with defective pixels is transcoded or up-converted, the interpolated pixels and their neighbors in the resulting video clip are somewhat blurred. The interpolation process is usually a weighted averaging process, i.e. calculated by determining an average RGB value of the neighboring pixels of an interpolated point, and assigning this average value to the interpolated point. Sometimes when the background of the defective pixels are very bright, then the defective pixels are difficult to notice, and may seem to be absent for periods of time.
RPN detection may typically be defined as a spatial-outlier-and-temporal-invariance detection problem. Performing such calculations at every pixel location throughout every location on a video uses unreasonable computational resources, especially in light of the fact that so few pixels are typically defective compared to the large number of pixels in a video, so it is not cost-effective to detect for RPN in such a manner.
Embodiments of the invention address these and other shortfalls of the prior art.